Poisson distribution

Poisson
Probability mass function

The horizontal axis is the index k, the number of occurrences. The function is only defined at integer values of k. The connecting lines are only guides for the eye.
Cumulative distribution function

The horizontal axis is the index k, the number of occurrences. The CDF is discontinuous at the integers of k and flat everywhere else because a variable that is Poisson distributed only takes on integer values.
Notation \mathrm{Pois}(\lambda)\,
Parameters λ > 0 (real)
Support k ∈ { 0, 1, 2, 3, ... }
PMF \frac{\lambda^k}{k!}\cdot e^{-\lambda}
CDF \frac{\Gamma(\lfloor k%2B1\rfloor, \lambda)}{\lfloor k\rfloor�!}\! for k\ge 0 or e^{-\lambda} \sum_{i=0}^{k} \frac{\lambda^i}{i!}\

(where \Gamma(x, y)\,\! is the Incomplete gamma function and \lfloor k\rfloor is the floor function)
Mean \lambda\,\!
Median \approx\lfloor\lambda%2B1/3-0.02/\lambda\rfloor
Mode \lceil\lambda\rceil - 1
Variance \lambda\,\!
Skewness \lambda^{-1/2}\,
Ex. kurtosis \lambda^{-1}\,
Entropy \lambda[1\!-\!\log(\lambda)]\!%2B\!e^{-\lambda}\sum_{k=0}^\infty \frac{\lambda^k\log(k!)}{k!}

(for large \lambda) \frac{1}{2}\log(2 \pi e \lambda) - \frac{1}{12 \lambda} - \frac{1}{24 \lambda^2} -
                    \frac{19}{360 \lambda^3} %2B O(\frac{1}{\lambda^4})
MGF \exp(\lambda (e^{t}-1))\,
CF \exp(\lambda (e^{it}-1))\,

In probability theory and statistics, the Poisson distribution (pronounced [pwasɔ̃]) (or Poisson law of small numbers[1]) is a discrete probability distribution that expresses the probability of a given number of events occurring in a fixed interval of time and/or space if these events occur with a known average rate and independently of the time since the last event. (The Poisson distribution can also be used for the number of events in other specified intervals such as distance, area or volume.)

Contents

History

The distribution was first introduced by Siméon Denis Poisson (1781–1840) and published, together with his probability theory, in 1837 in his work Recherches sur la probabilité des jugements en matière criminelle et en matière civile (“Research on the Probability of Judgments in Criminal and Civil Matters”).[2] The work focused on certain random variables N that count, among other things, the number of discrete occurrences (sometimes called “arrivals”) that take place during a time-interval of given length.

The first practical application of this distribution was done by Ladislaus Bortkiewicz in 1898 when he was given the task to investigate the number of soldiers of the Prussian army killed accidentally by horse kick; this experiment introduced Poisson distribution to the field of reliability engineering.[3]

Applications

Applications of Poisson distribution can be found in every field related to counting:

The distribution equation

If the expected number of occurrences in a given interval is λ, then the probability that there are exactly k occurrences (k being a non-negative integer, k = 0, 1, 2, ...) is equal to

f(k; \lambda)=\frac{\lambda^k e^{-\lambda}}{k!},\,\!

where

As a function of k, this is the probability mass function. The Poisson distribution can be derived as a limiting case of the binomial distribution.

The Poisson distribution can be applied to systems with a large number of possible events, each of which is rare. The Poisson distribution is sometimes called a Poissonian.

Poisson noise and characterizing small occurrences

The parameter λ is not only the mean number of occurrences E[k], but also its variance \sigma_k^2=E[k^2]-E[k]^2 (see Table). Thus, the number of observed occurrences fluctuates about its mean λ with a standard deviation \sigma_k =\sqrt{\lambda}. These fluctuations are denoted as Poisson noise or (particularly in electronics) as shot noise.

The correlation of the mean and standard deviation in counting independent discrete occurrences is useful scientifically. By monitoring how the fluctuations vary with the mean signal, one can estimate the contribution of a single occurrence, even if that contribution is too small to be detected directly. For example, the charge e on an electron can be estimated by correlating the magnitude of an electric current with its shot noise. If N electrons pass a point in a given time t on the average, the mean current is I=eN/t; since the current fluctuations should be of the order \sigma_I=e\sqrt{N}/t (i.e., the standard deviation of the Poisson process), the charge e can be estimated from the ratio \sigma_I^2/I. An everyday example is the graininess that appears as photographs are enlarged; the graininess is due to Poisson fluctuations in the number of reduced silver grains, not to the individual grains themselves. By correlating the graininess with the degree of enlargement, one can estimate the contribution of an individual grain (which is otherwise too small to be seen unaided). Many other molecular applications of Poisson noise have been developed, e.g., estimating the number density of receptor molecules in a cell membrane.

\Pr(N_t=k) = f(k;\lambda t) = \frac{e^{-\lambda t} (\lambda t)^k}{k!}

Related distributions

X_i \left|\sum_{j=1}^n X_j\right. \sim \mathrm{Binom}\left(\sum_{j=1}^nX_j,\frac{\lambda_i}{\sum_{j=1}^n\lambda_j}\right)
F_\mathrm{Poisson}(x;\lambda) \approx F_\mathrm{normal}(x;\mu=\lambda,\sigma^2=\lambda)\,

Occurrence

The Poisson distribution arises in connection with Poisson processes. It applies to various phenomena of discrete properties (that is, those that may happen 0, 1, 2, 3, ... times during a given period of time or in a given area) whenever the probability of the phenomenon happening is constant in time or space. Examples of events that may be modelled as a Poisson distribution include:

How does this distribution arise? — The law of rare events

In several of the above examples—such as, the number of mutations in a given sequence of DNA—the events being counted are actually the outcomes of discrete trials, and would more precisely be modelled using the binomial distribution, that is

X \sim \textrm{B}(n,p). \,

In such cases n is very large and p is very small (and so the expectation np is of intermediate magnitude). Then the distribution may be approximated by the less cumbersome Poisson distribution

X \sim \textrm{Pois}(np). \,

This is sometimes known as the law of rare events, since each of the n individual Bernoulli events rarely occurs. The name may be misleading because the total count of success events in a Poisson process need not be rare if the parameter np is not small. For example, the number of telephone calls to a busy switchboard in one hour follows a Poisson distribution with the events appearing frequent to the operator, but they are rare from the point of view of the average member of the population who is very unlikely to make a call to that switchboard in that hour.

Proof

We will prove that, for fixed \lambda, if

X_n \sim \textrm{B}(n,\lambda /n); \qquad Y\sim\textrm{Pois}(\lambda). \,

then for each fixed k

\lim_{n\to\infty}P(X_n=k) = P(Y=k).

To see the connection with the above discussion, for any Binomial random variable with large n and small p set \lambda=np. Note that the expectation E(X_n)=\lambda is fixed with respect to n.

First, recall from calculus

\lim_{n\to\infty}\left(1-{\lambda \over n}\right)^n=e^{-\lambda},

then since p = \lambda/n in this case, we have

\begin{align} \lim_{n\to\infty} P(X_n=k)&=\lim_{n\to\infty}{n \choose k} p^k (1-p)^{n-k} \\ &=\lim_{n\to\infty}{n! \over (n-k)!k!} \left({\lambda \over n}\right)^k \left(1-{\lambda\over n}\right)^{n-k}\\ &=\lim_{n\to\infty} \underbrace{\left[\frac{n!}{n^k\left(n-k\right)!}\right]}_{A_n} \left(\frac{\lambda^k}{k!}\right) \underbrace{\left(1-\frac{\lambda}{n}\right)^n}_{\to\exp\left(-\lambda\right)} \underbrace{\left(1-\frac{\lambda}{n}\right)^{-k}}_{\to 1} \\ &= \left[ \lim_{n\to\infty} A_n \right] \left(\frac{\lambda^k}{k!}\right)\exp\left(-\lambda\right) \end{align}

Next, note that

\begin{align} A_n &= \frac{n!}{n^k\left(n-k\right)!}\\ &= \frac{n\cdot (n-1)\cdots \big(n-(k-1)\big)}{n^k}\\ &= 1\cdot(1-\tfrac{1}{n})\cdots(1-\tfrac{k-1}{n})\\ &\to 1\cdot 1\cdots 1 = 1, \end{align}

where we have taken the limit of each of the terms independently, which is permitted since there is a fixed number of terms with respect to n (there are k of them). Consequently, we have shown that

\lim_{n\to\infty}P(X_n=k) = \frac{\lambda^k \exp\left(-\lambda\right)}{k!} = P(Y=k).

Generalization

We have shown that if

X_n \sim \textrm{B}(n,p_n); \qquad Y\sim\textrm{Pois}(\lambda), \,

where p_n=\lambda / n, then X_n\to Y in distribution. This holds in the more general situation that p_n is any sequence such that

\lim_{n\rightarrow\infty} np_n = \lambda.

2-dimensional Poisson process

Main article: Poisson process
P(N(D)=k)=\frac{(\lambda|D|)^k e^{-\lambda|D|}}{k!}

where

Properties

If X_i \sim \mathrm{Pois}(\lambda_i)\, follow a Poisson distribution with parameter \lambda_i\, and X_i are independent, then
Y = \sum_{i=1}^N X_i \sim \mathrm{Pois}\left(\sum_{i=1}^N \lambda_i\right)\,
also follows a Poisson distribution whose parameter is the sum of the component parameters. A converse is Raikov's theorem, which says that if the sum of two independent random variables is Poisson-distributed, then so is each of those two independent random variables.
\mathrm{E}\left(e^{tX}\right)=\sum_{k=0}^\infty e^{tk} f(k;\lambda)=\sum_{k=0}^\infty e^{tk} {\lambda^k e^{-\lambda} \over k!} =e^{\lambda(e^t-1)}.
D_{\mathrm{KL}}(\lambda\|\lambda_0) = \lambda_0 - \lambda %2B \lambda \log \frac{\lambda}{\lambda_0}.
P(X \geq x) \leq \frac{e^{-\lambda} (e \lambda)^x}{x^x}, \text{ for } x > \lambda
Similarly,
P(X \leq x) \leq \frac{e^{-\lambda} (e \lambda)^x}{x^x}, \text{ for } x < \lambda

Evaluating the Poisson distribution

Although the Poisson distribution is limited by

0 < f(k,\lambda) \le f(\lfloor \lambda \rfloor,\lambda) < 1,

the numerator and denominator of f(k,\lambda) can reach extreme values for large values of k or \lambda.

If the Poisson distribution is evaluated on a computer with limited precision by first evaluating its numerator and denominator and then dividing the two, then a significant loss of precision may occur.

For example, with the common double precision a complete loss of precision occurs if f(150, 150) is evaluated in this manner.

A more robust evaluation method is:

\begin{align} f(k,\lambda) &= \exp{(\ln{(f(k,\lambda))})} \\ &= \exp{(\ln{(\frac{\lambda^k \exp{(-\lambda)}}{k!})})} \\ &= \exp{(k\ln{(\lambda)} - \lambda - \sum_{i=1}^k \ln{(i)})}. \end{align}

Generating Poisson-distributed random variables

A simple algorithm to generate random Poisson-distributed numbers (pseudo-random number sampling) has been given by Knuth (see References below):

algorithm poisson random number (Knuth):
    init:
         Let L ← e−λ, k ← 0 and p ← 1.
    do:
         k ← k + 1.
         Generate uniform random number u in [0,1] and let p ← p × u.
    while p > L.
    return k − 1.

While simple, the complexity is linear in λ. There are many other algorithms to overcome this. Some are given in Ahrens & Dieter, see References below. Also, for large values of λ, there may be numerical stability issues because of the term e−λ. One solution for large values of λ is Rejection sampling, another is to use a Gaussian approximation to the Poisson.

Inverse transform sampling is simple and efficient for small values of λ, and requires only one uniform random number u per sample. Cumulative probabilities are examined in turn until one exceeds u.

Parameter estimation

Maximum likelihood

Given a sample of n measured values ki we wish to estimate the value of the parameter λ of the Poisson population from which the sample was drawn. To calculate the maximum likelihood value, we form the log-likelihood function

\begin{align} L(\lambda) & = \ln \prod_{i=1}^n f(k_i \mid \lambda) \\ & = \sum_{i=1}^n \ln\!\left(\frac{e^{-\lambda}\lambda^{k_i}}{k_i!}\right) \\ & = -n\lambda %2B \left(\sum_{i=1}^n k_i\right) \ln(\lambda) - \sum_{i=1}^n \ln(k_i!). \end{align}

Take the derivative of L with respect to λ and equate it to zero:

\frac{\mathrm{d}}{\mathrm{d}\lambda} L(\lambda) = 0 \iff -n %2B \left(\sum_{i=1}^n k_i\right) \frac{1}{\lambda} = 0. \!

Solving for λ yields a stationary point, which if the second derivative is negative is the maximum-likelihood estimate of λ:

\widehat{\lambda}_\mathrm{MLE}=\frac{1}{n}\sum_{i=1}^n k_i. \!

Checking the second derivative, it is found that it is negative for all λ and ki greater than zero, therefore this stationary point is indeed a maximum of the initial likelihood function:

\frac{\partial^2 L}{\partial \lambda^2} = -\lambda^{-2}\sum_{i=1}^n k_i

Since each observation has expectation λ so does this sample mean. Therefore it is an unbiased estimator of λ. It is also an efficient estimator, i.e. its estimation variance achieves the Cramér–Rao lower bound (CRLB). Hence it is MVUE. Also it can be proved that the sample mean is complete and sufficient statistic for λ.

Bayesian inference

In Bayesian inference, the conjugate prior for the rate parameter λ of the Poisson distribution is the Gamma distribution. Let

\lambda \sim \mathrm{Gamma}(\alpha, \beta) \!

denote that λ is distributed according to the Gamma density g parameterized in terms of a shape parameter α and an inverse scale parameter β:

g(\lambda \mid \alpha,\beta) = \frac{\beta^{\alpha}}{\Gamma(\alpha)} \; \lambda^{\alpha-1} \; e^{-\beta\,\lambda} \qquad \text{ for } \lambda>0 \,\!.

Then, given the same sample of n measured values ki as before, and a prior of Gamma(α, β), the posterior distribution is

\lambda \sim \mathrm{Gamma}(\alpha %2B \sum_{i=1}^n k_i, \beta %2B n). \!

The posterior mean E[λ] approaches the maximum likelihood estimate \widehat{\lambda}_\mathrm{MLE} in the limit as \alpha\to 0,\ \beta\to 0.

The posterior predictive distribution of additional data is a Gamma-Poisson (i.e. negative binomial) distribution.

Confidence interval

A simple and rapid method to calculate an approximate confidence interval for the estimation of λ is proposed in Guerriero et al. (2009). This method provides a good approximation of the confidence interval limits, for samples containing at least 15 – 20 elements. Denoting by N the number of sampled points or events and by L the length of sample line (or the time interval), the upper and lower limits of the 95% confidence interval are given by:

\lambda_{low}=\frac{(1-\frac{1.96}{\sqrt{N-1}}) N}{L}
\lambda_{upp}=\frac{(1%2B\frac{1.96}{\sqrt{N-1}}) N}{L}

The "law of small numbers"

The word law is sometimes used as a synonym of probability distribution, and convergence in law means convergence in distribution. Accordingly, the Poisson distribution is sometimes called the law of small numbers because it is the probability distribution of the number of occurrences of an event that happens rarely but has very many opportunities to happen. The Law of Small Numbers is a book by Ladislaus Bortkiewicz about the Poisson distribution, published in 1898. Some have suggested that the Poisson distribution should have been called the Bortkiewicz distribution.[10]

See also

Notes

  1. ^ Gullberg, Jan (1997). Mathematics from the birth of numbers. New York: W. W. Norton. pp. 963–965. ISBN 039304002X. 
  2. ^ S.D. Poisson, Probabilité des jugements en matière criminelle et en matière civile, précédées des règles générales du calcul des probabilitiés (Paris, France: Bachelier, 1837), page 206.
  3. ^ Ladislaus von Bortkiewicz, Das Gesetz der kleinen Zahlen [The law of small numbers] (Leipzig, Germany: B.G. Teubner, 1898). On page 1, Bortkiewicz presents the Poisson distribution. On pages 23-25, Bortkiewicz presents his famous analysis of "4. Beispiel: Die durch Schlag eines Pferdes im preussischen Heere Getöteten." (4. Example: Those killed in the Prussian army by a horse's kick.).
  4. ^ NIST/SEMATECH, '6.3.3.1. Counts Control Charts', e-Handbook of Statistical Methods, accessed 25 October 2006
  5. ^ McCullagh, Peter; Nelder, John (1989). Generalized Linear Models. London: Chapman and Hall. ISBN 0-412-31760-5.  page 196 gives the approximation and the subsequent terms.
  6. ^ Johnson, N.L., Kotz, S., Kemp, A.W. (1993) Univariate Discrete distributions (2nd edition). Wiley. ISBN 0-471-54897-9, p163
  7. ^ Philip J. Boland. "A Biographical Glimpse of William Sealy Gosset". The American Statistician, Vol. 38, No. 3. (Aug., 1984), pp. 179-183.. http://wfsc.tamu.edu/faculty/tdewitt/biometry/Boland%20PJ%20(1984)%20American%20Statistician%2038%20179-183%20-%20A%20biographical%20glimpse%20of%20William%20Sealy%20Gosset.pdf. Retrieved 2011-06-22. "At the turn of the 19th century, Arthur Guinness, Son & Co. became interested in hiring scientists to analyze data concerned with various aspects of its brewing process. Gosset was to be one of the first of these scientists, and so it was that in 1899 he moved to Dublin to take up a job as a brewer at St. James' Gate... Student published 22 papers, the first of which was entitled "On the Error of Counting With a Haemacytometer" (Biometrika, 1907). In it, Student illustrated the practical use of the Poisson distribution in counting the number of yeast cells on a square of a haemacytometer. Up until just before World War II, Guinness would not allow its employees to publish under their own names, and hence Gosset chose to write under the pseudonym of "Student."" 
  8. ^ Box, Hunter and Hunter. Statistics for experimenters. Wiley. p. 57. 
  9. ^ Massimo Franceschetti and Olivier Dousse and David N. C. Tse and Patrick Thiran (2007). "Closing the Gap in the Capacity of Wireless Networks Via Percolation Theory". IEEE Transactions on Information Theory. http://circuit.ucsd.edu/~massimo/Journal/IEEE-TIT-Capacity.pdf. 
  10. ^ Good, I. J. (1986). "Some statistical applications of Poisson's work". Statistical Science 1 (2): 157–180. doi:10.1214/ss/1177013690. JSTOR 2245435. 

References

External links
 
Continuous
Discrete